Automated procedure-determination and decision-generation

ABSTRACT

Methods and systems for automated monitoring are described herein. In various embodiments, image data of an enclosed area captured by an imaging device (102), audio data captured from the enclosed area by a sound sensor (104), and odor data captured from the enclosed area by an odor sensor (106), may be obtained by a computing device (HO). Based on the image, audio, and odor data, a determination (400) of an activity taking place within the enclosed area may be made. In various embodiments, one or more objects or persons in the enclosed area may be identified by performing image processing on the image data. The determined activity may be compared with corresponding data and parameters previously stored in a rules and schedules database (120). Based on the comparison and a decision generated by the rules and schedules database, an action may be selected to be performed.

BACKGROUND OF THE INVENTION

Facilities such as hospitals, assisted-living facilities (“ALF”), andnurseries cater to individuals that are dependent and in need of specialcare, such as children, elderly, and disabled people. These individualsoften need to be constantly monitored and taken care of. Aside frommonitoring occupants of a facility, it is also necessary to monitorevents and carry out health and safety-related procedures inside thefacility to ensure the well-being of facility residents or patients. Inhospitals, it would also be very useful to monitor the status of variousmedical equipment and devices, in addition to monitoring the status ofpatients in each hospital room without having to rely on medicalpersonnel to conduct those monitoring constantly. In a large hospital orhospice, ensuring consistent compliance and monitoring of the requiredand scheduled procedures could be challenging where there may be a largenumber of patients or residents that need to be tracked around theclock. An automated system for monitoring of those facilities andpersonnel's compliance with scheduled tasks could greatly enhancemonitoring efficiency, reduce or prevent accidents, improve compliance,and reduce overall costs associated with the operation of thosefacilities. Other facilities that could benefit from the application ofan automated monitoring system are warehouses, offices, stores, day carecenters, parks, zoos, and schools.

Monitoring systems using imaging devices such as closed circuittelevisions (“CCTVs”) are widely used for monitoring facilities.However, many of these monitoring systems are not automated and stillrequire human intervention (e.g., someone has to watch the CCTV feed) toconduct the monitoring of a facility and its personnel. Hence, thereexists a need for an automated system for monitoring residents,personnel, patients, occupants, actions, procedures, events, tasks, andobjects in a facility room or area.

U.S. Pat. App. No. 2013/0314522 discloses a patient monitoring systemthat monitors a patient, the patient's activity, and at least onemedical device and also discloses the use of a microphone to detectsounds. U.S. Pat. App. No. 2012/0075464 discloses a monitoring system todetermine actions taking place inside a monitored area.

The present disclosure offers an improvement over the aforementionedprior art by providing an automated monitoring system and method thatexecutes a corresponding action based on a determined procedure fromdetected and identified parameters and comparison with pre-stored dataand parameters.

SUMMARY OF THE INVENTION

The present disclosure relates to a method for automated monitoring ofone or more procedures comprising: obtaining, by one or more processors,image data captured by an imaging device of an enclosed area; obtaining,by the one or more processors, audio data captured from the enclosedarea by a sound sensor; obtaining, by the one or more processors, odordata captured from the enclosed area by an odor sensor; determining, bythe one or more processors based on the image, audio, and odor data, anactivity taking place within the enclosed area, wherein one or moreobjects or persons are identified by performing image processing on theimage data; comparing, by the one or more processors, the determinedactivity with corresponding data and parameters previously stored in arules and schedules database; and selecting, by the one or moreprocessors, an action to be performed based on the compared activity anda decision generated by the rules and schedules database.

In various embodiments, the method may further include obtaining motiondata captured by a motion sensor, wherein the determining is furtherbased on the motion data. In various embodiments, the method mayadditionally or alternatively include identifying motion of one or morepersons in the enclosed area based on the image data captured by theimaging device. In various versions, the determining may be furtherbased on the identified motion.

In various embodiments, the activity may violate one or more rules inthe rules and schedules database. In various versions, the action to beperformed may include raising an alert to the patient or to a caregiverthat the patient has violated the one or more rules. In variousversions, the one or more rules may include a rule that the patient issupposed fast for a predetermined time interval, and/or a rule that thepatient is supposed to be asleep during a predetermined time interval.

In various embodiments, the odor data may include the smell of urine,feces, or vomit in the enclosed area. In various versions, the action tobe performed may include raising an alert to prompt a caregiver tochange sheets or a diaper worn by a patient in the enclosed area. Invarious embodiments, the odor data may include the smell of smoke in theenclosed area.

In various embodiments, the method may further include performing imageprocessing on the captured image data to identify one or more persons inthe enclosed area. In various versions, the image processing may includeidentification of visual indicia worn by the one or more persons. Invarious versions, the image processing may include facial recognition ofthe one or more persons. In various embodiments, the method may furtherinclude performing object recognition on the captured image data toidentify medical equipment in the enclosed area.

The present disclosure also relates to a system for automated monitoringof one or more procedures taking place within an enclosed areacomprising: an imaging device to capture image data inside an enclosedarea; an odor sensor to detect one or more odors from the enclosed area;a rules and schedules database; and a computing device in communicationwith the imaging device, the odor sensor, and the rules and schedulesdatabase. The computing device may be configured to: perform imageprocessing on the image data to identify one or more objects and one ormore persons in the enclosed area; perform image processing on the imagedata to detect one or more movements of the one or more objects or bythe one or more persons; process and store contextual data relating tothe identified objects and persons, detected movements, and detectedodors; and analyze and correlate the contextual data with correspondingpre-stored data parameters in the rules and schedules database.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the disclosure, are incorporated herein to illustrateembodiments of the disclosure. Together with the description, they alsoserve to explain the principle of the disclosure. In the drawings:

FIG. 1 illustrates a block diagram of an automated monitoring systemaccording to one example embodiment of the present disclosure;

FIG. 2 illustrates an example of an automated monitoring system formonitoring an assisted living facility room;

FIG. 3 illustrates an example of an automated monitoring system foralerting a medical personnel regarding a scheduled patient visitation;

FIG. 4 illustrates a flowchart according to one example embodiment ofthe present disclosure;

FIG. 5 illustrates a flowchart for the determination of a proceduretaking place within an enclosed area according to one example embodimentof the present disclosure;

FIG. 6 illustrates a flowchart for the comparison between the determinedprocedure and the corresponding pre-stored parameters in the rules andschedules database; and

FIG. 7 illustrates a flowchart for the execution of a correspondingaction based on the compared procedure and a decision generated by thecomputing device using the rules and schedules database.

DETAILED DESCRIPTION OF EMBODIMENTS

The following are definitions of terms as used in the variousembodiments of the present disclosure.

The term “procedure” as used herein is used to collectively refer toactions, events, tasks such as a cleaning or maintenance task, andprocedures such as a medical procedure, test, or examination. The term“procedures database” as used herein refers to a database comprisingdata and pre-stored parameters corresponding to or relating to one ormore procedures, tasks, or events.

The term “room” as used herein refers to a space substantially orcompletely enclosed by one or more walls, a floor, and a ceiling thatcan be occupied by or populated with one or more people or objects. Theterm “hospital room” as used herein refers to a room in a medicalfacility, such as a hospital. The hospital room will typically includeone or more pieces of hospital equipment, devices, rooms, andfacilities, as well as patients, medical personnel, and visitors. Theterms “assisted-living facility room” and “ALF room” as used hereinrefer to a room in an ALF designed to provide various types of healthand medical-related services, personal care or personal assistanceservices such as nursing, medical, and housekeeping services. An exampleof an ALF is a housing facility for the disabled and the elderly. Theterm “nursery room” as used herein refers to a room or a facility whereinfants, toddlers, and children may be temporarily placed along with oneor more care providers. The nursery room may comprise at least onechild, a chaperone, and at least one nursery staff.

The term “medical setting” as used herein refers to a setting in a roomin a medical or health facility that includes particular objects andpersons of interest that provide context to the setting. For example, amonitoring system that detects the presence of a patient and an X-raytechnician that the monitoring system correctly identifies as beingscheduled to be present in the X-ray room of the hospital at or nearthat particular moment indicates to the monitoring system that an X-rayscan is about to be performed in the X-ray room involving thatparticular technician and patient.

The term “rules and schedules database” as used herein refers to adatabase comprising data and pre-stored parameters corresponding to orrelating to one or more rules, schedules, and corresponding actions ordecisions resulting from a comparison of those pre-stored data andparameters with detected, monitored, or measured one or more parameters.

The term “computing device” as used herein refers to any device capableof being programmed, performing mathematical and logical operations, ordisplaying and processing data and the results of any of thoseoperations. Computing devices may include, e.g., connected via one ormore buses, standard computing components such as one or moreprocessors, one or more types of memory (e.g., volatile memory such asRAM, non-volatile memory such as a hard drive), at least one computerprogram and/or operating system contained in the memory, a power supply,input/output devices, and so forth. Computing devices may come invarious form factors, including but not limited to desktop computers,laptop computers, tablet computers, smart phones, wearable computingdevices such as smart watches and smart glasses, set top boxes, and soforth.

The present disclosure relates to a method for automated monitoring ofone or more procedures comprising: obtaining, by one or more processors,image data captured by an imaging device of an enclosed area; obtaining,by the one or more processors, audio data captured from the enclosedarea by a sound sensor; obtaining, by the one or more processors, odordata captured from the enclosed area by an odor sensor; determining, bythe one or more processors based on the image, audio, and odor data, anactivity taking place within the enclosed area, wherein one or moreobjects or persons are identified by performing image processing on theimage data; comparing, by the one or more processors, the determinedactivity with corresponding data and parameters previously stored in arules and schedules database; and selecting, by the one or moreprocessors, an action to be performed based on the compared activity anda decision generated by the rules and schedules database.

The present disclosure also relates to a system for automated monitoringof one or more procedures taking place within an enclosed areacomprising: an imaging device to capture image data inside an enclosedarea; an odor sensor to detect one or more odors from the enclosed area;a rules and schedules database; and a computing device in communicationwith the imaging device, the odor sensor, and the rules and schedulesdatabase. The computing device may be configured to: perform imageprocessing on the image data to identify one or more objects and one ormore persons in the enclosed area; perform image processing on the imagedata to detect one or more movements of the one or more objects or bythe one or more persons; process and store contextual data relating tothe identified objects and persons, detected movements, and detectedodors; and analyze and correlate the contextual data with correspondingpre-stored data parameters in the rules and schedules database.

In one embodiment of the present disclosure, a method is directed to anautomated monitoring of one or more procedures comprising: identifyingobjects and persons and detecting movements using an imaging deviceconnected to a computing device, wherein the computing device isconnected to at least one database; capturing sounds in a room using asound sensor connected to the computing device; detecting smells usingan odor sensor; determining a procedure or event by comparing theidentified objects and persons, detected movements, captured sounds, anddetected smells with corresponding pre-stored parameters in a proceduresdatabase and other databases such as a sounds database, odor database,personnel database, and objects database; comparing the determinedprocedure with corresponding pre-stored parameters in a rules andschedules database; and executing a corresponding action based on thecompared procedure and a decision generated by the rules and schedulesdatabase. In an example embodiment, data relating to persons, objects,movements, sounds, and smells are stored in a personnel database,objects database, motion database, sounds database, and odor database,respectively.

In another embodiment of the disclosure, a system is provided forautomated monitoring of one or more procedures comprising: an imagingdevice for identifying objects and persons and detecting movements; asound sensor for capturing sounds from the objects and persons; an odorsensor for detecting smells; a procedures database; a rules andschedules database; a sounds database; an odor database; and a computingdevice connected to the imaging device, sound sensor, and odor sensorfor processing and storing data relating to the identified objects andpersons, detected movements, captured sounds, detected smells and foranalyzing and comparing said data with corresponding pre-storedparameters and data in the databases that include the proceduresdatabase, rules and schedules database, personnel database, objectsdatabase, sounds database, and odor database.

Depending on the financial capability or budget of a hospital orfacility, a facility owner or administrator may choose to implement themethod and system of the present disclosure only in critical facilityrooms and areas such as the intensive care unit or the operating room.

In one example embodiment of the present disclosure illustrated in FIG.1, automated monitoring system 100 may include an imaging device 102, asound sensor 104, an odor sensor 106, and/or a motion sensor 108. Thesecomponents may be in communication with a computing device 110. Thecomputing device 110 may include, among other standard computingcomponents that are not depicted in FIG. 1, a communications module 112.The computing device 110 may also be in communication (e.g., via one ormore networks 126) with a sounds database 114, an odor database 116, aprocedures database 118, a rules and schedules database 120, a personneldatabase 122, and/or an objects database 124. The databases 114, 116,118, 120, 122, and/or 124 may reside within the vicinity of thecomputing device 110 (e.g., in the same room, or within the samecomputing device), or may be stand-alone computing devices connected tothe computing device 110 through the cloud or a network 126. Thedatabases 114, 116, 118, 120, 122, and/or 124 may each be implemented ona separate computing system, or they may be collectively distributedacross one or computing systems, such as blade servers of a server farm.

As shown in the example illustrated in FIG. 1, a main server 128 may bein communication with the network 126. In addition, the main server 128may be in communication with a security monitoring device 130, apersonnel monitoring device 132, and a supervisor monitoring device 134.The security monitoring device 130, personnel monitoring device 132, andsupervisor monitoring device 134 can be any of a laptop computer,desktop computer, cell phone, tablet computer, a set top box, a PDA, orother common form factors for computing devices.

Imaging device 102 may be any device capable of capturing, recording, orstoring an image, video, and other forms of digital or electronicrepresentation of an object or a scene. Imaging devices incorporatedinto various embodiments may come in various forms. These forms arewell-known and are available commercially. Commonly used imaging devicesare semiconductor charged-coupled devices (“CCD”) or active pixelsensors in complementary metal-oxide-semiconductor (“CMOS”). Examples ofimaging devices may include digital cameras, video cameras,closed-circuit televisions (“CCTVs”), webcams and smart cameras.

Sound sensor 104 may be any device capable of capturing, recording, orstoring a digital representation of a sound. Sound sensor 104 may takevarious forms, most of which are well-known and are availablecommercially. Many of these sound sensors are microphone-based. Examplesof microphone-based sound sensors include piezoelectric microphones,MEMS microphones, condenser microphones, fiber optic microphones,electret condenser microphones, dynamic microphones, ribbon microphones,carbon microphones, laser microphones, and liquid microphones. Sounddata captured by sound sensor may, in some embodiments, be stored insounds database 114.

Odor sensor 106 may be any device or component capable of capturing,recording, and/or storing odor data representative of a sensed odor.Odor sensors may alternatively be referred to as “electronic noses.”Various types of odor sensors 106 appropriate for use in the presentdisclosure are known and available commercially. Odor sensors 106 mayoperate based on various techniques, such as the use of conductingpolymers, polymer composites, metal-oxide semiconductors, quartz crystalmicrobalance, surface acoustic wave, mass spectrometry, and ultra-fastgas chromatography. Data collected by these odor sensors 106 may be usedin conjunction with other technology such as artificial intelligence,deep learning, fuzzy logic, artificial neural networks, patternrecognition, etc. for analyzing odor-related data. Captured odor datamay be stored in the odor database 116.

Motion sensors 108 that may be used in accordance with the disclosureare those that are based on infrared, optical, microwave, ultrasonic,acoustic, or tomographic sensors, and those that fall under the categoryof occupancy sensors. The sensors used in accordance with the disclosuremay be of passive or dynamic type, or a combination of both. Passiveinfrared sensors work by detecting heat movement. They comprise apyroelectric sensor designed to detect infrared radiation radiated by amoving body. When the passive infrared sensor detects body heatmovement, the sensor may activate any lighting system connected to it.

Ultrasonic sensors, which are similar to radar sensors, are based on theDoppler-shift principle. An ultrasonic sensor transmits high frequencysound waves in a monitored area and detects reflected wave patterns.When it detects continuously changing reflected pattern, the automatedmonitoring system 100 takes that to mean that someone is occupying aroom and light inside the room is switched on. If the reflected patterndoes not change for a given period of time, the sensor assumes the roomis unoccupied and the light is turned off.

Microwave sensors works in a similar way except that they transmit highfrequency microwaves. Microwave sensors offers the advantage in thatthey are very sensitive and can have a detection range as far as 1,500feet. The monostatic type can be used to define a detection area basedon the microwave beam that it emits and can have a detection range up to400 linear feet. The user may vary the beam configuration, e.g., long,narrow, short, or oval shape, depending on the intended application. Thebistatic version offers a longer detection range, up to 1,500 linearfeet, but it does not allow one to define the area of detection asprecisely as a monostatic sensor. While many motion detectors are ableto detect up to 15 feet, tomographic-based motion sensors offer thecapability of covering large areas and the ability to penetrate mostwalls and obstructions and to detect multiple locations, rather thanjust at the location of the transmitter.

To minimize the incidences of false alarms, a combination of differenttypes of motion sensors 108 may be used, such as a combination ofinfrared and microwave motion sensors. For example, the infraredsensor's detection of a movement can be used to trigger the microwavesensor, which requires a higher threshold to trip, so that an alarm oralert is triggered only when the microwave sensor also detects motionfollowing the infrared sensor's detection of a movement.

The communications module 112 may be a standalone device or a componentof a device that facilitates communication—e.g., sending and receivingof commands, triggers, notifications, prompts, acknowledgments,information, messages, forms, and various types of data such as video,text, and audio—between, for example, the computing device 110 and anyone of the sensors such as the imaging sensor 102, sound sensor 104, andodor sensor 106, or between computing device 110 and other computingdevices and systems depicted in FIG. 1. Direct communication among thevarious sensors, instead of just between the computing device 110 andthe various sensors, may also take place in accordance with thedifferent embodiments of the present disclosure. Thus, in one exampleembodiment of the present disclosure, the computing device 110 and eachsensor may have their own communications modules to allow communicationwith each other and other devices and networks. As used herein, whendevices are described as being “in communication with” each other, thatmay refer to communication using various wireless technologies (e.g.,Wi-Fi, Bluetooth, ZigBee, coded light, directed audio, etc.) and/orvarious types of physical connections or channels such as copper wires,optical fibers, Ethernet, etc.

In some embodiments, the computing device 110 may initially send one ormore commands, via the communications module 112, to one or more sensorsto wake them up from a sleep state or other low power states. Each ofthe triggered sensors may then transmit an acknowledgment to thecomputing device 110 via its own communications module (not depicted inFIG. 1) to signal that it has been activated. Alternatively, theactivation of a first sensor by the computing device 110 may be followedby the triggering of the other sensors by the first sensor (instead ofby the computing device 110) through the sensors' own communicationsmodules. In an example embodiment, each sensor sends or receives data toand from the computing device 110, which may then transmit the datathrough the communications module 112, via the network 126, to the mainserver 128 or to at least one of the databases 114, 116, 118, 120, 122and 124. In another embodiment of the disclosure, the sensors maydirectly transmit or exchange data via their own communications moduleto or with any of the databases 114, 116, 118, 120, 122 and 124 or themain server 128 through the network 126, instead of first sending thedata to the computing device 110.

In accordance with the various embodiments of the present disclosure,communications module 112 may include any transmitter or receiver usedfor Wi-Fi, Bluetooth, infrared, NFC, radio frequency, cellularcommunication, visible light communication, Li-Fi, WiMAX, ZigBee, andother forms of wireless communication. Alternatively, the communicationsmodule 112 may be connected to network 126 via a physical channel suchas a USB cable, fiber optics, or other wired forms of communication.

In accordance with various aspects of the present disclosure, objectsand persons may be detected and identified using image data captured byimaging device 102. Object and person detection may be accomplishedbased on the captured image data using various techniques, such aspattern recognition, image recognition, image registration, objectrecognition, facial recognition, feature extraction, color recognition,optical character recognition, motion analysis covering tasks such asegomotion, tracking, and optical flow, pose estimation, machine vision,machine learning, content-based image retrieval, shape recognition,artificial intelligence, neural networks, and other methods of imageprocessing as well as other types of computer vision-based technology.Data relating to hospital personnel, such as images of physicians,nurses, security, and maintenance staff may be stored in the personneldatabase 122. Data relating to hospital equipment, devices, and medicalsupplies may be stored in the objects database 124.

In an example embodiment of the present disclosure, the imaging device102 may be used along with a motion sensor 108 installed in front of ahospital room, for example, to allow for automatic opening of doors, todetect the presence or identity of anyone who wishes to enter (orenters) the room. In some embodiments, one or more motion sensors 108may be strategically installed in various locations inside a room suchas a hospital or an ALF room. When the motion sensors 108 detect motion,they may activate other sensors such as sound sensor 104, odor sensor106, and imaging device 102. For example, when the motion sensors 108detect motion, the automated monitoring system 100 may turn on thelights in the room (if initially switched off) to permit timely andaccurate detection and recognition of any person inside the room usingimage data captured by imaging device 102.

In various embodiments, the scheduled procedures, tasks, events,maintenance, and visitations in hospitals, ALFs, or nursery schools maybe encoded and stored in the procedures database 118 and/or the rulesand schedules database 120. For example, the procedures database 118 maystore requested patient visitations, patient examinations and othermedical procedures, hospital room maintenance tasks, scheduled securitychecks, etc... A computer program may then be used to perform thescheduling of each requested task or procedure depending on the urgencyof need, availability of one or more medical personnel, availability ofthe hospital facility required for a medical procedure, availability ofneeded medical supplies, etc. Once the schedules for these tasks andprocedures have been fixed, the automated monitoring system 100 thenencodes these schedules tasks, events, and procedures into the rules andprocedures database 118.

The schedules for the various procedures may be stored in the rules andschedules database 120, while pending tasks may be stored in theprocedures database 118. Any previously-scheduled tasks which have notyet been performed and not yet rescheduled may be assigned as pendingtasks and may be stored as such in the procedures database 118.

In some embodiments, the rules and schedules database 120 may prescribethe order in which each scheduled tasks is to be monitored, howfrequently to monitor, what the alternative action should be in case atask or part of a task was not performed, etc. For example, one or morecomputing devices associated with the automated monitoring system 100may detect if the scheduled administration of a medicine was performedbefore the patient's meal, and may notify the medical personnel if thatparticular medicine was supposed to have been administered after meals.Also, one or more computing devices associated with the automatedmonitoring system 100 may notify the maintenance department via email,phone call, and/or text message if a maintenance personnel forgot toclean one area in one of the hospital rooms. One or more computingdevices associated with the automated monitoring system 100 may alsonotify the appropriate department of an unscheduled event if, forexample, the automated monitoring system 100's odor sensor 106 in one ofthe hospital rooms suddenly detects a foul smell inside the room. Othertypes of databases may also be used in conjunction with the proceduresdatabase 118 and the rules and schedules database 120, such as anaccounts and billings database (not depicted).

As an example, suppose a hospital rule regarding noise and visitationhours is enforced every day past midnight in every hospital room. In oneexample embodiment, the automated monitoring system 100 may beconfigured to monitor and detect for a set time interval, such asbetween midnight and 7 am, sounds that correspond to loud noises, apatient falling from a bed or in the bathroom, people talking,screaming, loud music playing, banging noises, alarms, bells, toysounds, sirens, and loud sounds coming from a radio, television,computer, laptop, etc. In some embodiments, the automated monitoringsystem 100 may be configured to only raise an alert when sound levelsabove a certain threshold are detected. In this way, a sound that wouldnormally not disturb or awaken a resting or sleeping patient may stillbe detected by the automated monitoring system 100 but would not causean alert or notification to be triggered and transmitted to the hospitalstaff or security.

In various embodiments, certain smells such as the smell of urine,feces, vomit, medicinal smells, cauterizing smells, and/or smoke insidea room can also be monitored and detected by the present disclosureusing an odor sensor 106 and a smoke detector. In some embodiments, theautomated monitoring system 100 may be configured to raise one or morealerts in response to one or more detected odors, e.g., in conjunctionwith detection of other data points such as persons and/or medicalequipment in the area. For example, if the smell of feces, urine, and/orvomit is detected in an enclosed area, such as a patient's private room,the automated monitoring system 100 may raise an alert (e.g., send anemail or text message, or make an audible announcement over a publicannouncement system) to prompt one or more caregivers to clean sheets,change a diaper, or take other appropriate responsive action.

The presence of visitors or persons other than the patient in a hospitalroom may be detected using the imaging device 102, optionally inconjunction with at least one motion sensor 108. In some embodiments,every patient, medical personnel, maintenance staff, security personnel,administrative personnel, and every other hospital employee within thehospital premises may be required to wear a uniform that includes somesort of visual indicia for identifying them. This visual indicia maycome in various forms, such as a large-sized barcode, and may be placedat various positions on the person, such as in front and/or at the backof a patient's robe or a doctor's uniform. In some embodiments, thevisual indication may also include a person's complete name, department,and position, e.g., adjacent to the barcode.

Alternatively, the identification indicia on the hospital robe oruniform may include a pattern or set of symbols, which may be unique toeach hospital employee or patient, to make it easy for the automatedmonitoring system 100 of the present disclosure to distinguishindividuals who, for example, are still in the room past visitationhours. In some embodiments, every hospital visitor may be required towear an identification indicia, such as a barcode pin, RFID tag, or aremovable visitor identification sticker that can be placed both infront and the back of a visitor's clothes. Thus, visitors who stay inthe hospital rooms beyond visitation hours can easily be tracked andappropriately notified or alerted when visitation hours have ended.Requiring all visitors to wear any of these identification indicia mayalso enable the automated monitoring system 100 to identify any personand thus prevent any unauthorized entry to any room in the hospital. Forexample, the automated monitoring system 100 may prevent a patient'svisitor from entering a hospital laboratory or an MRI room, unlesspreviously authorized by the hospital administration and unless theauthorization has been entered into the automated monitoring system 100.The authorization may be issued by a main attending physician or ahospital administrator via his or her hospital-issued portable device134 or by an automated monitoring system 100 administrator. Theautomated monitoring system 100 administrator can be, for example, thehead of the hospital's IT department who work closely with the hospitaldirector or administration office.

The automated monitoring system 100 of the present disclosure may alsouseful for monitoring a room such as a hospital or assisted livingfacility room for identifying prohibited activities. For example, themonitoring system 100 may monitor patients who are required to fast fora certain time period, e.g., prior to a fasting blood panel. Usingobject recognition, odor detection, and/or radiation detection, themonitoring system 100 may detect when, for example, visitors bring inprohibited items or devices that may interfere with the operation ofhospital equipment, or that could exacerbate a patient or resident'sexisting ailment or condition. Various types of radiation detectors,such as a radio wave detector, are known and available commercially. Inthe case of cigarette smoking detection, the automated monitoring system100 may also include the use of a smoke detector, in addition to an odorsensor 106 and an imaging device 102.

In another example, the monitoring of a scheduled medical personnelvisitation or patient check-up may be conducted by the automatedmonitoring system 100 by first running through the scheduled patienttests or examinations or other scheduled tasks for that day that werepreviously encoded into the procedures database 118 and the rules andschedules database 120. The automated monitoring system 100 may thenconfirm via the rules and schedules database 120 other details relatingto the scheduled procedure such as the hospital room to which thepatients is to be transported, the type of procedure to be performed onthe patient, and the medical personnel assigned to perform theprocedure. The automated monitoring system 100 may also look up thepersonnel database 122 for other needed information such as the name,department, title, position, and contact information of hospitalemployees scheduled to show up for a patient visitation.

The automated monitoring system 100 may then send notification to theassigned medical personnel to remind them ahead of time of the scheduledvisitation and other procedures or tasks based on the information storedin the rules and schedules database 120 and other databases. Theautomated monitoring system 100 may repeatedly the reminders to theassigned medical personnel until each of the assigned medical personnelhas confirmed their receipt of the reminder. When at least one of theassigned medical personnel has failed to respond to the automatedmonitoring system 100, say, within five minutes of the scheduled patientvisitation, the automated monitoring system 100 may transmit anotification to the concerned medical personnel over the hospital'spublic broadcast system or paging system. If the missing medicalpersonnel still fails to reply to the public notification after a givenperiod, the automated monitoring system 100 may note the failure of themissing medical personnel to reply and send a message to the missingmedical personnel, appropriate department, supervisor, or administrationoffice that the medical personnel has failed to show up for a scheduledpatient visitation. The automated monitoring system 100 may also send arequest, for example, to the main attending physician, appropriatedepartment, nursing station, or to the administration office for asubstitute medical personnel to replace the missing medical stafforiginally assigned to attend to the scheduled patient visitation.

The system-generated reminder for the scheduled patient visitation maybe sent by the automated monitoring system 100 to hospital-issuedportable devices 130, 132, and 134 assigned to each medical personnel,the notification being sent within a preset time, for example, at leastfifteen minutes or half-an-hour before the scheduled visitation,depending on the urgency of the procedure. The same hospital-issuedportable devices 130, 132, and 134 assigned to each medical personnelmay also be used by the doctors, for example, to transmit requests forthe scheduling of a medical procedure, task, examination or laboratorytest for a patient by the automated monitoring system 100.

In some embodiments, detection of the entry of the assigned medicalpersonnel into the scheduled patient room may be performed by theautomated monitoring system 100 by scanning, for example, a barcode wornor embossed on the medical personnel's uniform, or by reading aradio-frequency identification (“RFID”) tag worn by the medical staffusing an RFID reader. Confirmation of the identity of the medicalpersonnel assigned to conduct a patient visitation for that time may beperformed via, for example, facial or pattern recognition. The medicalstaff patient visitation may be partially or entirely recordeddigitally. If the identified medical personnel is not the one assignedto conduct the patient check-up, e.g., the medical personnel was alast-minute replacement, the automated monitoring system 100 maycommunicate this to the substitute medical personnel over a microphonepreferably located inside the hospital room, and the automatedmonitoring system 100 may also communicate with the medical stafforiginally assigned to handle the patient visit to confirm from him orher that the identified replacement will be conducting the patientcheck-up at the scheduled time.

In various embodiments, in conjunction with the procedure and systemdescribed above, the automated monitoring system 100 may assign a jobcode for each scheduled task or procedure, such as a four-number orfive-number code. Each of these assigned job codes may then betransmitted to the medical personnel or maintenance staff scheduled toperform a task or procedure. In addition, each hospital employee mayhave a unique identification code and/or a hospital-issued smart cardthat identifies them to the automated monitoring system 100. In oneembodiment of the present disclosure, the medical or maintenancepersonnel assigned to perform a procedure may first be required to enterhis or her identification code using a numeric keypad that is connectedto the automated monitoring system 100 and which is located at the roomentrance. When the automated monitoring system 100 recognizes that theidentification code is valid, the medical personnel may then be asked toenter the job code assigned to a scheduled procedure that was previouslysent to the medical personnel. This tells the automated monitoringsystem 100 that the medical personnel has arrived in the scheduledhospital room to perform the procedure. After the medical personnel hascompleted the procedure, he or she may enter his or her security codeand the job code again, which informs the automated monitoring system100 that the task or procedure has been completed. The monitoring system100 may then record the time and day during which the procedure wasperformed, and also may record the identity of the medical personnel whoperformed the task.

In a further embodiment, a piece of specialized equipment used by amedical personnel during a patient visitation may have a button or otherinterface that when actuated, automatically transmits (wirelessly or viaphysical connections) a notification to the automated monitoring system100 that the particular hospital device is to be used by a medicalstaff. Pressing the same button a second time or pressing a separate“complete” or “end” button transmits another notification to theautomated monitoring system 100 that the use of the specific hospitaldevice or equipment has been completed and the device or equipment isnow in an “off” state. The scheduled hospital procedure may includechecking the vital signs of a patient, intravenous fluid levels, andhospital equipment status, providing the patient's medicines, monitoringthe intake of those medicines, performing or assisting in performingpatient baths, such as a sponge bath, for incapacitated patients, and soforth. These procedures may be recorded by the automated monitoringsystem 100 and the digital recording stored in one of the hospital'sdatabases for a set duration, after which the recording may be deletedto recover electronic storage space.

The automated monitoring system 100 may also be used to detect, e.g.,via image data captured by the imaging device 102, the status of thevarious medical equipment and devices inside a hospital room, includingmedical supplies, such as the current level of oxygen gas andintravenous fluid. If the automated monitoring system 100 detects abelow threshold value for any of those medical equipment, devices orsupplies, the automated monitoring system 100 may send a notification tothe medical staff and/or maintenance personnel.

Following these steps, the automated monitoring system 100 may then senda message to the hospital's accounting and billing department, forexample, that relays at least some of the following information to theaccounting database: the type and model number of the device orequipment used at a particular time and room, patient name, age, primaryphysician assigned (if any), duration and number of times of use of thehospital device and equipment, and the name of the medical staff thatused the equipment, amount of medication administered, a number offungible hospital consumables (e.g., IV bags, needles, sponges,bandages, etc.) used, among others. The appropriate fees for the use ofthe equipment and any corresponding doctor's fees may be chargedautomatically by the automated monitoring system 100 to the patient'saccount.

In still another example, the automated monitoring system 100 may beused to monitor a scheduled test or examination of a patient who is tobe transported into another room or in a different area of a hospital,e.g., physician room, X-ray room, MRI room, physical therapy room,rehabilitation department, or surgery room. Again, the automatedmonitoring system 100 may first send reminders to the assigned medicalstaff regarding a scheduled patient examination in another room orfacility, e.g., at least half-an-hour, before the scheduled patientexamination. The same or similar notification procedure as described indetail above may be used. In a similar way as previously described, theautomated monitoring system 100 may detect the entry of the hospitalstaff assigned to transport the patient to another room or facilitywithin the hospital and also identifies the medical personnel conductingthe test or examination in the other room or facility.

Thus, the automated monitoring system of the present disclosure may actas a multifunctional and comprehensive monitoring system that monitorsdifferent events, tasks, procedures, locations, personnel, schedules,device and equipment status, maintenance requests, bill payments,patient discharge dates, etc. For example, in addition to the examplesprovided above, the automated monitoring system 100 can also be used todetect and identify the equipment used in the test or examination, suchas X-ray or MRI, and then transmit the corresponding information to thehospital's accounts and billing department database to record theprocedure performed on the patient and automatically charge thepatient's account accordingly. If medicines, consumables such assyringe, intravenous fluid, blood from a blood bank, and othernon-reusable medical items were administered during a scheduled hospitalvisit, the automated monitoring system 100 will notify the appropriatedepartment such as the hospital pharmacy, blood bank, requisitions,accounting and billing, etc. that those medical items, among others,were used or consumed during the scheduled hospital visit. This helpsensure that the hospital is always amply stocked with those medicalitems. In some embodiments, the automated monitoring system 100 may alsoautomatically place an order for the specific medical items that arerunning low based on the preset threshold relating to the minimum numberor amount of a medical or maintenance objects that should always bereadily available for the entire hospital staff.

In still another embodiment of the disclosure depicted in FIG. 2, theautomated monitoring system 100 may be used to monitor the performanceof a scheduled regular hospital or assisted living facility roommaintenance. In this embodiment, the entrance of a cleaning staff 200 ina room is detected by scanning the staff's barcode on his uniform usingan imaging device 102 (not depicted in FIG. 2) or by the maintenancestaff's entering his employee code and appropriate job code into anumeric keypad (not depicted) located in front of a room. In anotheralternative, the cleaning staff wears an RFID tag 202 which is then readby an RFID sensor/reader 204 strategically located inside the facilityroom. As shown in FIG. 2, the bathroom inside the facility's room to becleaned by the janitor includes a motion detector 206 to detect theentry or presence of one or more persons. Sensors, including the motiondetector 206, for recording the motion and location of the cleaningpersonnel allows the detection and monitoring of the cleaning of variousareas of the room including a bathroom or washroom inside a facility orhospital room. Again, the entire room cleaning procedure may be recordedand stored digitally by the automated monitoring system 100 and thenuploaded to a database or server in the facility maintenance department.The recorded video may be used, for example, in determining if thescheduled room maintenance has been performed, if all areas of the roomhave been cleaned and the sheets replaced, and whether the cleaning andother maintenance tasks, such as replacing the bathroom toilet paper 208and other toiletries, have been performed properly.

In another embodiment of the disclosure, the automated monitoring system100 may be used to monitor activities and sleeping schedules of theoccupants inside an ALF.

For example, the automated monitoring system 100 may detect noises,laughter, and recognize the faces of the ALF residents via the imagerecognition system. Based on this detected data, the automatedmonitoring system 100 may determine that ALF residents are still insidethe recreation room of the facility even though it is already past theresidents' bedtime(s). In another case, the automated monitoring system100 may detect the presence of an elderly resident who has wondered intoan unauthorized area of the facility at 2 am. The automated monitoringsystem 100 may then send an alert to the facility staff, e.g., thenursing or security personnel, informing them about the situation. Theautomated monitoring system 100 may also automatically switch off thelights in the residents' rooms at the scheduled residents' bedtime whenit fails to detects any motion within, say, fifteen minutes from thebeginning of the set bedtime. In some embodiments, the automatedmonitoring system 100 may automatically play a soothing music or aresident's favorite music, say, for the first ten minutes from the setbedtime. The automated monitoring system 100 may also send anotification to the facility staff if continuous motion away from thebed is detected, such as when the resident is walking back and forth inhis or her room past bedtime.

The automated monitoring system 100 of the present disclosure may alsobe used to record certain activities of the residents inside the ALF.For example, the residents' relatives may request that the automatedmonitoring system 100 regularly transmit, for example, by the end ofeach day, recorded videos of the residents while they are engaged inactivities such as exercising and eating. This allows the family orrelatives of the resident to monitor the well-being of the resident andensure that the resident is eating properly and exercising regularly.

When the automated monitoring system 100 fails to detect the performanceof a scheduled task such as a scheduled doctor visitation or a roommaintenance within a given time, such as half-an-hour past the scheduledtask or event, the automated monitoring system 100 may transmit anotification, preferably through the hospital's paging system first, toone or more of the following: nursing station, primary physician,assisting primary physician, hospital department head or office such asEENT, radiology, neurology, pediatrics, otolaryngology, gynecology, orthe hospital administration office. If the primary physician, forexample, fails to reply to the hospital's paging system, such as whenthe primary physician is outside the hospital, the automated monitoringsystem 100 may then send an email and/or a text message directly to theprimary physician's cellphone or other portable devices 134. Until anappropriate reply has been received from any of the called or pagedparty, the automated monitoring system 100 will keep sending thenotification repeatedly at regular intervals, such as every ten minutes.When the primary physician, for example, has sent the appropriate replyto the automated monitoring system 100, such as by texting or emailingthe reply “notification received,” the automated monitoring system maystop sending the alerts and the doctor can then proceed to the hospitalroom to perform the scheduled patient visitation.

In some cases, as illustrated in FIG. 3, a first physician 300 who isalerted regarding a scheduled patient visitation may instead request asecond physician 302 to conduct the scheduled visitation within aspecific time range. Also shown in FIG. 3 are a microphone 304 forrecording sounds and an odor sensor 306 for detecting smells. An imagingdevice 308 may also be used for identifying persons, medical suppliessuch as the intravenous fluid bag 310, and equipment such as the ECGmonitor 312, and for detecting movement inside the hospital room. Insuch a case, the first physician 300 may request the automatedmonitoring system 100 to capture a video and sound recording of thehospital room during the scheduled visitation by the second physician302. After some time, the first physician 300 may then confirm if thepatient visitation has been conducted by the second physician 302 byrequesting the automated monitoring system 100 to download a videorecording of the requested patient visitation to the first physician300's device 314. This may allow the first physician 300 not only toverify if the patient visitation had already performed by the otherphysician 302, but may also allow the first physician to verify if allthe required procedures scheduled for the patient had been completed andif the procedures were performed properly. This would be particularlyuseful if the first, more senior physician 300, for example, is thesecond physician 302's supervising physician, such as when the secondphysician 302 is working for his medical internship, residency, orfellowship at the hospital under the supervision of the first physician300. The supervising physician 300 may then later use the downloadedrecorded video to discuss the patient's status with the second physician302, and to also review the performance of the second physician 302being trained and supervised. The same or similar procedure may also beused in cases where the scheduled task or event involves other medicalstaff, maintenance personnel, security staff, and administrativepersonnel.

FIG. 4 illustrates a preferred method of the present disclosure. Atblock 400, the automated monitoring system 100 may determine a proceduretaking place within an enclosed area by determining contextual datarelated to the enclosed area. For example, the automated monitoringsystem 100 may identify contextual data such as objects and personspresent in the area, along with corresponding sounds and/or odors sensedwithin the enclosed area. As noted above, identification of objects andpersons may be accomplished, for instance, by performing imageprocessing on image data captured by an imaging device 102.

At block 402, the contextual data and/or the procedure determined basedon the contextual data may be compared with the corresponding pre-storedparameters in the rules and schedules database 120. At block 404, theautomated monitoring system 100 may execute a corresponding action basedon the compared procedure and a decision generated by the rules andschedules database 120 based on the compared procedure. After executingthe system-prescribed action based on the system-generated decision, themethod may return to block 400, at which the automated monitoring system100 repeats the process of monitoring any detected one or moreprocedures.

FIG. 5 illustrates an example method of determining a procedure(operation 400) illustrated in FIG. 4. At block 500, the automatedmonitoring system 100 may identify objects and persons using the imagingdevice 102 and detecting movements by the motion sensor 108 (step 500).Persons and objects may be identified by comparing acquired datarelating to persons and objects with pre-stored data (e.g., personneland hospital equipment photographs) and parameters in the personneldatabase 122 and objects database 124, respectively. In someembodiments, motion detection may be performed based on imaging datacaptured by the imaging device 102, rather than (or in conjunction with)based on data captured by the motion sensor 108. For example, a sequenceof frames of a video captured by imaging device 102 may be used todetect user motion.

At block 502, the automated monitoring system 100 may detects soundsusing the sound sensor 104. At block 504, the automated monitoringsystem 100 may detect odors using the odor sensor 106. At block 506, theautomated monitoring system 100 may determine a procedure to beperformed, e.g., by correlating the identified objects and persons,detected movements, captured sounds, and detected odors with thecorresponding pre-stored data and parameters in the procedures database118. For example, suppose the automated monitoring system 100 identifiesa doctor and an MRI scanner inside a hospital room. By correlating thedata relating to the identified doctor and MRI scanner with thepre-stored data and parameters in the procedures database 118, say,under persons and objects category, the automated monitoring system 100may determine that an MRI procedure is scheduled to be performed insidethe hospital room on that day and time. After the automated monitoringsystem 100 determines the procedure, the determined procedure may thenbe correlated with corresponding pre-stored data and parameters in therules and schedules database 120 at block 402 of FIG. 4.

FIG. 6 illustrates a preferred method of the comparison between thedetermined procedure and the corresponding pre-stored parameters in therules and schedules database 120 (block 402) illustrated in FIG. 4.After determining a procedure at block 400 of FIG. 4, at block 600, theautomated monitoring system 100 may verify if the determined procedurecorresponds to a scheduled procedure, task, or event. If the determinedprocedure corresponds to a scheduled procedure, at block 602, thedetermined procedure may be further verified for compliance with therules stored in the rules and schedules database 120. Alternatively, ifat block 600 the determined procedure does not comply with the rules andschedules, at block 604, the computing device 110 or another componentmay determine a corresponding alternative decision and action from therules and schedules database 120. For example, suppose a janitor wasdetected to have entered an activity room in an ALF at 9:00 AM. Theautomated monitoring system 100 may verify from the rules and schedulesdatabase 120 that there is a scheduled cleaning task at 9:00 AM in theactivity room. If the automated monitoring system 100 determines thatthe janitor has not entered the activity room's bathroom by the end ofthe cleaning period, the automated monitoring system 100 may concludesthat the bathroom of the activity room has not been cleaned. Theautomated monitoring system 100 may send a notification to the janitoror his supervisor to notify him or her that the bathroom had not beencleaned as scheduled.

FIG. 7 illustrates an example method of the execution of a correspondingaction based on the compared procedure and a decision generated by therules and schedules database 120 (operation 404). After the computingdevice 110 determines the corresponding decision and action from therules and schedules database 120 (block 604), at block 700, theautomated monitoring system 100 may check if the action requires anotification. If a notification is required, at block 702, the automatedmonitoring system 100 may sends a notification to the appropriateindividuals or departments. Afterwards, at block 704, the automatedmonitoring system 100 may verify if the system-required actionnecessitates activating one or more devices and equipment. If device orequipment activation is required, at block 706, the automated monitoringsystem 100 may activate, switches on, and/or otherwise enable thecorresponding one or more devices and equipment. Examples of suchactivation of devices and equipment are automatically locking doors,turning off lights, adjusting room temperature, and closing windows.Afterwards, the automated monitoring system 100 continues to monitor oneor more procedures within the enclosed area.

While several inventive embodiments have been described and illustratedherein, those of ordinary skill in the art will readily envision avariety of other means and/or structures for performing the functionand/or obtaining the results and/or one or more of the advantagesdescribed herein, and each of such variations and/or modifications isdeemed to be within the scope of the inventive embodiments describedherein. More generally, those skilled in the art will readily appreciatethat all parameters, dimensions, materials, and configurations describedherein are meant to be exemplary and that the actual parameters,dimensions, materials, and/or configurations will depend upon thespecific application or applications for which the inventive teachingsis/are used. Those skilled in the art will recognize, or be able toascertain using no more than routine experimentation, many equivalentsto the specific inventive embodiments described herein. It is,therefore, to be understood that the foregoing embodiments are presentedby way of example only and that, within the scope of the appended claimsand equivalents thereto, inventive embodiments may be practicedotherwise than as specifically described and claimed. Inventiveembodiments of the present disclosure are directed to each individualfeature, system, article, material, kit, and/or method described herein.In addition, any combination of two or more such features, systems,articles, materials, kits, and/or methods, if such features, systems,articles, materials, kits, and/or methods are not mutually inconsistent,is included within the inventive scope of the present disclosure.

All definitions, as defined and used herein, should be understood tocontrol over dictionary definitions, definitions in documentsincorporated by reference, and/or ordinary meanings of the definedterms.

The indefinite articles “a” and “an,” as used herein in thespecification and in the claims, unless clearly indicated to thecontrary, should be understood to mean “at least one.”

The phrase “and/or,” as used herein in the specification and in theclaims, should be understood to mean “either or both” of the elements soconjoined, i.e., elements that are conjunctively present in some casesand disjunctively present in other cases. Multiple elements listed with“and/or” should be construed in the same fashion, i.e., “one or more” ofthe elements so conjoined. Other elements may optionally be presentother than the elements specifically identified by the “and/or” clause,whether related or unrelated to those elements specifically identified.Thus, as a non-limiting example, a reference to “A and/or B”, when usedin conjunction with open-ended language such as “comprising” can refer,in one embodiment, to A only (optionally including elements other thanB); in another embodiment, to B only (optionally including elementsother than A); in yet another embodiment, to both A and B (optionallyincluding other elements); etc.

As used herein in the specification and in the claims, “or” should beunderstood to have the same meaning as “and/or” as defined above. Forexample, when separating items in a list, “or” or “and/or” shall beinterpreted as being inclusive, i.e., the inclusion of at least one, butalso including more than one, of a number or list of elements, and,optionally, additional unlisted items. Only terms clearly indicated tothe contrary, such as “only one of” or “exactly one of” or, when used inthe claims, “consisting of,” will refer to the inclusion of exactly oneelement of a number or list of elements. In general, the term “or” asused herein shall only be interpreted as indicating exclusivealternatives (i.e. “one or the other but not both”) when preceded byterms of exclusivity, such as “either,” “one of” “only one of,” or“exactly one of” “Consisting essentially of” when used in the claims,shall have its ordinary meaning as used in the field of patent law.

As used herein in the specification and in the claims, the phrase “atleast one,” in reference to a list of one or more elements, should beunderstood to mean at least one element selected from any one or more ofthe elements in the list of elements, but not necessarily including atleast one of each and every element specifically listed within the listof elements and not excluding any combinations of elements in the listof elements. This definition also allows that elements may optionally bepresent other than the elements specifically identified within the listof elements to which the phrase “at least one” refers, whether relatedor unrelated to those elements specifically identified. Thus, as anon-limiting example, “at least one of A and B” (or, equivalently, “atleast one of A or B,” or, equivalently “at least one of A and/or B”) canrefer, in one embodiment, to at least one, optionally including morethan one, A, with no B present (and optionally including elements otherthan B); in another embodiment, to at least one, optionally includingmore than one, B, with no A present (and optionally including elementsother than A); in yet another embodiment, to at least one, optionallyincluding more than one, A, and at least one, optionally including morethan one, B (and optionally including other elements); etc.

It should also be understood that, unless clearly indicated to thecontrary, in any methods claimed herein that include more than one stepor act, the order of the steps or acts of the method is not necessarilylimited to the order in which the steps or acts of the method arerecited.

1. A method for automated monitoring comprising: obtaining, by one ormore processors, image data captured of an enclosed area by an imagingdevice; obtaining, by the one or more processors, audio data capturedfrom the enclosed area by a sound sensor; obtaining, by the one or moreprocessors, odor data captured from the enclosed area by an odor sensor;determining, by the one or more processors based on the image, audio,and odor data, an activity taking place within the enclosed area,wherein one or more objects or persons are identified by performingimage processing on the image data; comparing, by the one or moreprocessors, the determined activity with corresponding data andparameters previously stored in a rules and schedules database; andselecting, by the one or more processors, an action to be performedbased on the compared activity and a decision generated by the rules andschedules database.
 2. The method of claim 1, further comprisingobtaining, by the one or more processors, motion data captured by amotion sensor, wherein the determining is further based on the motiondata.
 3. The method of claim 1, further comprising identifying, by theone or more processors, motion of one or more persons in the enclosedarea based on the image data captured by the imaging device.
 4. Themethod of claim 3, wherein the determining is further based on theidentified motion.
 5. The method of claim 1, wherein the activityviolates one or more rules in the rules and schedules database, andwherein the action to be performed comprises raising an alert to apatient or to a caregiver that the patient has violated the one or morerules.
 6. The method of claim 5, wherein the one or more rules comprisea rule that the patient is supposed to fast for a predetermined timeinterval.
 7. The method of claim 5, wherein the one or more rulescomprise a rule that the patient is supposed to be asleep during apredetermined time interval.
 8. The method of claim 1, wherein the odordata comprises the smell of urine, feces, or vomit in the enclosed area.9. The method of claim 8, wherein the action to be performed comprisesraising an alert to prompt a caregiver to change sheets or a diaper wornby a patient in the enclosed area.
 10. The method of claim 1, whereinthe odor data comprises the smell of smoke in the enclosed area.
 11. Themethod of claim 1, further comprising performing, by the one or moreprocessors, image processing on the captured image data to identify oneor more persons in the enclosed area.
 12. The method of claim 11,wherein the image processing comprises identification of visual indiciaworn by the one or more persons.
 13. The method of claim 11, wherein theimage processing comprises facial recognition of the one or morepersons.
 14. The method of claim 1, further comprising performing, bythe one or more processors, object recognition on the captured imagedata to identify medical equipment in the enclosed area.
 15. A systemfor automated monitoring of one or more procedures taking place withinan enclosed area comprising: an image data input for receiving imagedata captured inside an enclosed area by an imaging device; an odorinput for receiving a signal from an odor sensor arranged to detect oneor more odors from the enclosed area; a rules and schedules database;and a computing device in communication with the imaging device, theodor sensor, and the rules and schedules database, the computing deviceto: perform image processing on the image data to identify one or moreobjects and one or more persons in the enclosed area; perform imageprocessing on the image data to detect one or more movements of the oneor more objects or by the one or more persons; store contextual datarelating to the identified objects and persons, detected movements, anddetected odors; and analyze and correlate the contextual data withcorresponding pre-stored data parameters in the rules and schedulesdatabase.